Consumer self-profiling gui, analysis and rapid information presentation tools

ABSTRACT

An input and processing system allows user input information such as user affinity to efficiently determine user characteristics from content as well as novel input of commands such as copy/paste on a small mobile device screen among other computing devices. A client/server is also made more efficient due to the enhanced gathering of information. Also disclosed is a system for rapid presentation of information such as barcodes at barcode scanners. A system can request multiple barcodes, coupons or the like by detecting a signal from a scanner that information presented has been read.

CROSS-REFERENCE TO RELATED APPLICATION(S)

The present application is a continuation of U.S. patent applicationSer. No. 13/707,581, filed Dec. 6, 2012, now U.S. Pat. No. ______, whichclaims the benefit of and priority to the following U.S. ProvisionalApplications: U.S. Provisional Application No. 61/724,863 filed Nov. 9,2012; U.S. Provisional Application No. 61/603,216 filed Feb. 24, 2012;U.S. Provisional Application No. 61/597,136 filed Feb. 9, 2012; and U.S.Provisional Application No. 61/567,594 filed Dec. 6, 2011, which areherein incorporated by reference in their entirety.

The following U.S. Provisional Applications are also herein incorporatedby reference in their entirety: U.S. Provisional Patent Application No.61/493,965 filed Jun. 6, 2011; U.S. Provisional Patent Application No.61/533,049 filed Sep. 9, 2011; U.S. Provisional Patent Application No.61/506,601 filed Jul. 11, 2011; and U.S. Provisional Patent ApplicationNo. 61/683,678 filed Aug. 15, 2012.

In addition, the following applications are incorporated by reference intheir entirety: U.S. patent application Ser. No. 13/490,444 filed Jun.6, 2012; U.S. patent application Ser. No. 13/490,449 filed Jun. 6, 2012;U.S. patent application Ser. No. 13/490,447 filed Jun. 6, 2012; andInternational Patent Application No. PCT/US12/41178 filed Jun. 6, 2012.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1-4 illustrates an embodiment of the consumer self-profiling toolsusing angular direction in accordance with an embodiment of thedisclosed technology.

FIG. 5 illustrates an embodiment of the consumer self-profiling toolsusing command zones in accordance with an embodiment of the disclosedtechnology.

FIGS. 6-8 illustrate an embodiment of the consumer self-profiling toolsusing multiple data feeds in accordance with an embodiment of thedisclosed technology.

FIG. 9 illustrates an embodiment of the consumer self-profiling tools asapplied to collaborative filtering results.

FIG. 10 illustrates an embodiment of persona driven in-game ads.

FIG. 11 illustrates an embodiment of a rapid information presentationtool in accordance with an embodiment of the disclosed technology.

FIG. 12 illustrates a block diagram of a user's computing device inaccordance with an embodiment of the disclosed technology;

FIG. 13 illustrates one embodiment of a networked computing system usedin implementing the disclosed technology;

FIGS. 14-16 illustrate briefly the technology discussed in theincorporated patent applications including brand sorting, tying brandsto marketing data, a GUI for a brand sorter, an audience engine, servingads based upon a user's profile, the creation of a user's profile basedon brands, and an interest graph among other concepts.

APPENDICES

Appendix A has a description of technologies described in theincorporated applications.

Appendix B has exemplary computer code that can be used to implement theSwote™ Tools described.

DETAILED DESCRIPTION Introduction

There is currently a need for a new type of relevant information thatcan be provided to users of computer-based devices that is not availableby previously existing means. Previously, consumers were satisfied withrelevance information from search engines and social media. Searchengines use links between web pages to determine relevance and socialmedia uses links between people to determine social relevance.

One new kind of relevant information that is in demand is “right now”relevance (e.g., the answer to the question, “Which pub should I go tonear my current location and with the specific group of friends I amwith?”). Unlike relevance that is “forever” or “overall” like that foundwith search engines, this new kind of relevance information needs lotsof other information determine, which is often not easily input by theuser (especially via mobile devices).

As will be explained in further detail below, one embodiment of thedisclosed technology provides a convenient way for a user to enterinformation that can be used to provide “right now” relevanceinformation. The information is gathered by monitoring how a user reactsto the depiction of a number of items of content such as “ads” and“brands.” As used herein a brand can be virtually anything that evokessome sort of positive or negative reaction from the user and which canbe correlated to some demographic characteristic. As explained in theincorporated patent applications, a brand may include, but is notlimited to, a logo, trademark, animation, text, movies, movie clip,movie still, TV shows, books, musical bands or genres, celebrities,historical or religious figures, geographic locations, colors, patterns,occupations, hobbies or any other thing that can be associated with somedemographic information. For instance any thing that can be broadlyaccepted or recognized by a plurality of users can be a brand. Suchexamples could be Huggies™ brand diapers, Copper River Salmon™,Microsoft™ software, a picture of Tom Cruise, a picture of a frame fromone of Tom Cruise's movies, a musical band name, a musical band albumcover, a famous picture such as the picture from Time™ magazinecelebrating victory in WWII in which a sailor is kissing a woman, apicture of a house in the country, a picture of a Porsche™ car, apicture of a smiley face, locations (France, Virgin Islands) etc. Inaddition, brands can be abstract ideas such as “World Peace” and “Savethe Whales”, political ideologies such as Republican” or other conceptsabout which a user may have an opinion can also be brands. As usedherein, the term “ad” is to be interpreted broadly and can includepromotional materials, rebates, consumer notices, content, political orreligious materials, coupons, advertisements (including pushadvertisements), various kinds of recommendations (such asproduct/service recommendations, content/media recommendations), offers,content (movies/TV shows) and other information that a user may which toreceive.

By determining an individual's reaction to one or more brands in an easyand intuitive way, information such as characteristics which includedemographic characteristics and interests, can be inferred about theperson. For example, people who like Nike™ brand shoes are typicallyyounger individuals who have in interest in fitness. This likelydemographic information can be used to offer goods and services to theindividual.

In the incorporated patent applications, a user's profile of theirlikely characteristics is referred to as an “Advertar”™, which is has anumber of actual or desired demographic and interest tags associatedwith it as well as associated statistical probabilities. In oneembodiment of this technology, Advertars can be presented to merchantswho offer the advertar goods or services. However, the owner of theadvertar may remain anonymous to the merchants.

Overview of Swote™ Consumer Self-Profiling Tools

To allow a user to provide information about themselves withoutanswering a number of long and tedious questions, a user of thedisclosed technology can simply swipe content such as icons on a displayscreen. Input from swiping content on a display in order to vote on howthe user feels about it is referred to herein as “Swote” input (e.g.swipe+vote input). Swote tools discussed below provide: 1) a user a GUI(Graphical User Interface) with the ability to select/input a pluralityof commands through an intuitive interface on any device; and 2) amechanism to allow a user to profile herself by self-curating unpreparedcontent. From the ranking/categorization of like, dislike, numerical,relative ranking of content, or other methods of ranking/groupingcontent via the user input tools, the user can create/augment anadvertar profile regarding her interests/preferences. For example, auser can create/augment an interest graph-based profile via these toolswhich reflect data about her such as herinterests/preferences/demographic data. From this data associated withthe user's profile or advertar, an analysis may be conducted and contentwhich includes items such as brands, ads, offers, coupons,products/services, news, data feeds, pictures or any other data filesetc. can be delivered to the user. This matching may be done asdiscussed in the incorporated applications by using distance/personavectors.

Swote input may be performed by a gesture which moves an image ofcontent on a display screen through a GUI. Swote GUI tools may focus onthe 1) user's angular direction of an initial user point of interactionwith the brand/content and the final end point of interaction with thecontent e.g., the angular direction between the point where the userfirst touches the brand/content and the point in where the user ends thetouch; or 2) the use of command zones in which the user places herfinger/cursor or the content.

Swote GUI interaction and analysis may occur for self-curated,unprepared content. The unprepared content includes content, in whichpreviously has not been formatted for processing by taxonomies, analysisfor meaning, integration profiles etc. or whose formatting is not knownto the entity that will analyze the data. This unprepared content mayinclude information with no or few meaningful related data (e.g., tags).Some examples include: any content that may be displayed to a user suchas news stories on a news website, hyperlinks, pictures as well asindividual pieces of content such as user selected words, emails in herinbox, URLs, webpages, brands, ads, local files (local or remote) suchas Microsoft Word documents through a file browser, music, videos,social media information such as friend profiles, user comments,products (pictures of the product, text, product numbers, SKU codes),users (Facebook, other advertars, celebrities) etc. These can all be,individually or simultaneously, the subject of Swote commands. Variousother commands such as copy, paste, send, delete etc. can also beincorporated to be executed separately or with in combination with aSwote input. As such, Swote tools offer collection and the determinationtools applicable to any data, which is then formatted for use such asanalysis of the data's relevance to the user by those with no priorknowledge of the meaning of the data in the data's original context orthe data's meaning to a user.

Tagging/assigning characteristics such as demographic characteristicsand interests and assigning statistical probabilities to content thathas received Swote input and tags as well as updating the likelycharacteristics defined by a user's advertar can be accomplished viaparsing, semantic processing, natural language processing, context,image analysis (facial/product/geographical recognition), taxonomies,analysis of the information (content) source such as a URL/URI/datalinks and its demographics and related files, analysis ofsocial/interest graphs/advertar of the user or other users, userpersonal information, user location, user activities, information fromuser computing devices, social media trends, marketing data andstatistical probabilities as discussed in the incorporated applicationsas well as combinations of these tools.

In one embodiment of the disclosed technology, a user provides inputabout a brand by performing a Swote gesture an icon that represents thebrand on a screen. The Swote gesture may be detected on a touchsensitive screen or using any other type of input device that can recorda user's gesture such as a mouse or air gestures (e.g., MicrosoftKinect™), in response to the presentation of a particular items ofcontent. Content may be presented visually (icons on a display screen)aurally (in music or the spoken word), textually (brand names embeddedin other content) or via other means where the user can recognize thecontent and perform a gesture or otherwise provide a response to thecontent. In one embodiment, the direction of the gesture is indicativeof whether the user likes or dislikes the content. However, how thegesture is performed may, also be interpreted to have a meaning aboutthe content.

Exemplary Swote Tools Embodiment

FIGS. 1-4 illustrate one embodiment of a computing device 101 that isenabled to use Swote input. Device 101 maybe connected by a network 1310to a content provider 118, (that provide content such as webpage 100),audience engine 120 (that may store profiles) and the internet (that mayprovide a connection to any network, coupons sites, offer networks, adnetworks, brands, advertisers and other devices/users 122. Device 101 isillustrated with memory 1204, wired or wireless network connection 1214,display 1206 (e.g., touch screen), processor(s) 1202, input 1206 andoutput 1210. Network 1310 may be any network such as a local areanetwork as well as the Internet.

In the example shown, a webpage 100, such as an unprepared webpage fromYahoo.com has a number of content items 104 displayed. The user performsa Swote gesture on one or more of the content items displayed by moving(optionally overriding a previously assigned drag and drop command) animage of the content item either up (to indicate a like) or down (toindicate a dislike). In the embodiment shown, content 104 representingsitcom stars of the 80's receives a Swote up input by user by placing apointing device such a mouse or finger or air gesture 106 on the imageof the content. The user then moves their finger up in the desireddirection depending on how the user is performing the Swote gesture. Inthe example shown, the content is moved in an upward direction asindicated by the arrow 110.

In one embodiment, the webpage 100 or other content is prepared(pre-configured) by the webpage author such as a content provider withSwote input enabling features such as gestures and indicia. In addition,characteristic tags (demographic characteristics and interest tags) andassociated statistical probabilities can be assigned to the content, thewebsite can also be configured to determine related tags via a taxonomyas well as display the available commands to a user and indicia toindicate which command was selected. Finally, the website can beconfigured to assign transmission instructions and associatedinformation such as sending the discussed data above as well as a user'sadvertar ID to a remote device. In an embodiment discussed below, webbrowser plug-ins, web browsers, operating systems, and/or applications(including mobile applications) on local or remote devices, mayautomatically do some or all of these tasks for any unprepared content.Statistical probabilities and weights as discussed below may be eitherindividual or indexed to the population or a combination.

As illustrated, a tag 124 (such as an affinity tag) is assigned by auser gesture to the content 104. The information included in the tag 124may reference, at least a portion of the content, identity or networkaddress of the content and related information (see below). The taginformation is sent to a remote server such as audience engine 120 todetermine the meaning of the Swote gesture. As used herein, an affinitytag can be any indication of a user's liking/disliking by suchindications as a thumbs up/down, numerical rating, a relationship suchas ownership or lack of ownership, would buy again or not would buyagain etc.

Content on, the webpage 100 may be prepared for Swote input through theuse of user input in the following operations. First, the user directsinput such as a mouse hover input or a “touch and hold” input on adisplay such as a touch screen display on or near content 104 which hasa picture and text. In response to the input, it is determined thatrelated information 122 about both the picture and text is available.

Related content information 122 may be located by a variety of ways. Inone embodiment, the content on the webpage is related to other contentvia the webpage ancestry. For instance, a still from a movie next to orotherwise in close proximity to the news story 204 can be related byexamining the webpage 100 DOM (Document Object Model) and determiningsaid relation. Information may be also be related if the same or similartags (e.g., HTML tags) are associated to the information. In anotherembodiment, related information is information that has been frequentlyassociated together by other users. In yet another embodiment, relatedinformation may be information that share the same or similartext/graphics/sounds/authors/domain of origin/topics/categories/titlesor have been related by a taxonomy.

In another embodiment, related information can be found by examininglinks such as hyperlinks. The picture of content 104 links to a URL ofthe full news story in which further text and pictures are displayed tothe user. This data which is linked to content 104 by hyperlinks isrelated information. In addition, data links from file managers, socialmedia, or any other link can be used as related information. Relatedinformation may be also found through following one link to anotherlink. For instance, hyperlinking to the story that content 104 is on andhyperlinking to a second story linked to that, increase a relatedinformation lineage.

Here this may be that both the picture and text in content 104 haverelated information as determined by the URL addresses they are from.(e.g., perhaps both from yahoo.com) The user may select any number ofSwote input content items for simultaneous input of Swote gestures.Here, the user desires to select both and does so by clicking on boththe text and picture.

In response to verifying that there are data links that referencerelated information, indicia such as shading the content 104 can bedisplayed to the user indicating the content is available for Swoteinput. A menu displaying available commands such as a Swote up input,Swote down input, copy paste etc. can be displayed. Here, the userSwotes up on the content. In response to this gesture, a Swote up tag isassigned to the content. In addition one or more of the identity of thecontent, at least a portion of the content, a location pointer (networkaddress, global address), tag 124 and any related information maybeanalyzed locally or transmitted for remote analysis and integration intothe user's profile. In another embodiment, the content does not need tobe linked to receive a Swote input, for the content item to be shadedand/or swotable. For instance, a picture without related information mayreceive a Swote up gesture. In response, in one embodiment a portion ofthe picture, the identity of the picture and/or a Swote up tag may beanalyzed.

From the analysis of the Swote content, various characteristics can beadded to a user profile. Here, the content 104 is determined by naturallanguage and image processing to be about an 80's TV show. Content 104can be associated with the characteristics of an 80's TV fan, and one ormore associated statistical probabilities through marketing, otheruser's interest graphs and taxomonies. For example, if the user inputs aSwote up gesture on the content 104, there is a 90% probability she isan 80's TV fan. In addition, a taxonomy and marketing data for this TVshow can be used to supplement the user profile. For instance, in theexample shown, the user may be statistically likely to have demographiccharacteristics of being a male, a certain age X, makes a certain amountof income and is likely has 50% interest in pizza during the late nighthours etc. The incorporated patent applications discuss this further.The assignment of tags and probabilities to content may occur remotelyor locally.

In another embodiment, a web browser plug-in, application or operatingsystem (e.g., through the file manager) may be instructed toautomatically enable content such as pictures, text, icons, ads, couponsand movies in a webpage or other application such as a file browser,media selection window (e.g., movie selection screen on Netflix™ or RokuBox™), contacts/friends window on a mobile device, a mobile deviceapplication etc., to receive Swote input (assign commands relevant toproviding a reaction to the content) as well as optionally assign tagsand probabilities to the content.

In FIG. 1, brand/content item 104 is shown with the user's hand or mousecursor 106 above it. Here, the user clicks the mouse 106 and selects thecontent item 104 at starting point 112. This selection of the contentcan be via various methods such as drag and drop via a mouse, finger,stylus, web cam movement of a body part, voice, air gesture, keyboardetc. Once the content is selected, pre-defined commands can be executedby the user which may include assigning a numerical rating, a relativerating (information relative to other pieces of content or information),a “like”, “dislike”, “neutral” rating or other rating systems to thecontent.

As illustrated in FIG. 1, command assignment based on a gesture likemovement in an the angular direction from the starting point 112 to anend point 114 at which the user releases the content on the screen fromselection 114. In another embodiment, this may be executed without acontinuous selection and movement of the content is defined by aninitial input point and a final input point.

In this embodiment, two commands are available depending on the angulardirection where the user moves the brand/content. If the user moves thebrand/content in a substantially up direction as indicated by arrow 110,then a “thumbs up” preference is recorded and if a substantially downangular direction is input, a “thumbs down” preference is recorded.

Here, the user moves the content 104 substantially up within apredefined range associated with a “Swote up” input command whilemaintaining selection of content 104 in relation to the starting point112 designated by the user's click or other input on the content. Asthis occurs, icon 108 is displayed as the command that will be selected.Alternately, and also optional, the original place on the web page thatcontent 104 occupied before being moved can be shown with a differentrepresentation of the content 104.

In response to the user deselecting the content item 104, the picture204 “snaps” back into its original starting point as illustrated in FIG.2. Icon 108 is shown to the user indicating a thumbs up preference wasassigned to content 104. Icon 108 may be shown next to, on top of or inany other position in relation to content 104.

In another embodiment, the content remains static during the user inputof angular movement while icon 108 indicating which of a number ofcommands or inputs will be given or applied to the content as it isbeing moved by the user, is overlaid or displayed in proximity to thecontent. In yet another embodiment, as the user drags the content, amenu is displayed that displays a plurality of the available commands (aportion or all simultaneously). In another embodiment, the user maypress a key e.g., the SHIFT key to see a menu of commands that areavailable during input of angular movement. Here a thumbs up and downsign is displayed and the user may drag the content in the direction ofone of these signs to execute the command. In one embodiment, in whichmultiple pieces of selected content with some commands in common andwith other commands not, holding the SHIFT key will display only thecommands in common to the user.

Appendix B includes reprehensive source code executable by a processorto implement a Swote input selection on an item of content. As discussedabove, any content is available for selection by the user. In oneembodiment, as illustrated in Appendix B, the position of the user'scursor may select the content, such as the nearest image or anchor tagthat is near/under the cursor's current position on a display. In asimilar manner, the operations in Appendix B that determine the contentselected can also be used to determine the related information.

In FIG. 3, the user selects and moves content 104 substantially down inrelation to starting point 112 as indicated by arrow 110. The user thendeselects the content 104 at point 114, which inputs the command of athumbs down preference in relation to content 104 based on angulardirection of the input from 112 to 114. Additionally, before the userdeselects brand/content item (or a representation of the item) 104 asseen in FIG. 4, icon 108 is displayed which displays a thumbs downindicia. In other embodiments, the action to be taken (thumbs up etc.)and the command (the vote was a thumbs up) maybe displayed and orfeedback given to the user via vibration, color or through otherindications.

Once the user has completed a Swote gesture, a Swote tag 124 such as anaffinity tag, the Swote content, as well as any other data that islinked to the content (related information) can be analyzed to determineor update likely demographic information about the user. This mayinclude associated URLs, text, tags, the type of content, thecontent/category of data/users based that the domain on which thecontent typically attracts etc. The analysis can be performed locally orremotely and used to create or update a user's profile.

Information that may be useful in gleaning likely demographicinformation from the Swote operation which may also be analyzed caninclude contextual Swote information such as the time, data,user/advertar identity, device type, OS type, account information,location of the user when performing the Swote gesture, proximity ofbusinesses, brands, friends and other information to the user, browserhistory and other commands/tags associated to content 204 such as:metadata, page source code, URL and related URLs, related files, dataappearing on the same page or file system and related data such as linksto other files, results from collaborative filtering, taxonomy datadetermined from analysis of the above data such as relatedcategories/interests and marketing data/statistical probabilitiesrelated to the data above.

In one embodiment, a user's profile or advertar is created or updatedtaking into account the information that can be gleaned from the user'sSwote input of a particular content item. Information gleaned from thepositive or negative Swote input of a content item (e.g., an affinitytag is associated to the content) is incorporated into the user'sadvertar by, for example, creating new tags with an associateddemographic probability/interests and associated probabilities orsupplementing existing tags. Various methods can be used to weight orotherwise consider tags in the analysis.

A webpage author may enable the Swote tools via a web browser extensionor other tools such as a browser plug-in that manually or automaticallyparses, determines or interprets the data within the webpage, stream,etc. that is content enabled for interaction with by the user. Alsodetermined may be the type of content, the relevant commands based onthe type of content or even relevant to the specific content itself suchas adding sports specific commands for content about sports. Software inthe form of executable instructions are stored in a non-transitorycompute readable memory such as memory 1204, which cause a processor todetect the selection of a brand image or content item and a gesturedirected to a reflected brand or items of content according to one ormore gesture rules. Once the user completes the Swote gesture on thebrand/content item, the software then either computes likelycharacteristics associated with the Swote command or sends anotification of the Swote command and related information to a remoteprocessing unit that does the analysis.

Swote tools may be used on any computing device and combinations ofcomputing devices connected to each other and executed with a finger,stylus, mouse or other input method. Ads and advertar relatedinformation can be input and output to these devices from third partycomputing devices based on the information entered via the Swote toolsand profiles. These Swote tools will operate/interoperate on anydevice/platform without “uglifying” the information presentation such asby placing unsightly “like” buttons on pictures.

Other Swote Tool Embodiments

In other embodiments, different commands may be input or displayed andthen selected for input depending on the angular direction the userselects in relation to the starting point. For instance, the commandscould be: copy, paste, vote up, assign metadata tag X of a Swotegesture, vote down, delete, email, SMS, post to social media,right/left/middle mouse input, keyboard input, hover input, stylusinput, select, add to profile, add to interest graph, bookmark, save,interact with, move content to X destination (e.g., a certain row in abrand sorting screen), find out more about, help, visualize pastselection, set location, taking a picture of displayed content etc. Insome embodiments, a numerical or other ranking/voting may be assigned tothe content. Different degrees of rank can be assigned by draggingcontent more or less (voted more in the up direction to like more etc.)distance from starting point 112 as illustrated in FIG. 1. In oneembodiment, if a user inputs Swote input on content such as a Swote upinput on an ad, a similar ad may appear to replace the first ad. Thesecommands may be executed by a Swote gesture in any combination.

Other commands/combinations are possible such as displaying a numericalslider bar for voting/ranking, assignment to a “read later list”,bookmark list, emailing the data or links to the data to herself orfriends, execute related searches, open a link to a wiki page, mail thedata to a specific contact, post to Facebook, request the data be shownto her later-for instance on a different or same device e.g., the datain the form of an ad shown on her IP TV screen, show relatedinformation, show data from a specific source such as data X but from aGoogle Shopping search, copy to clipboard, etc.

In one embodiment, a Swote input may be used to input commands duringthe playing of an audio or visual file. A first Swote input maydesignate that the file is desired/undesired (e.g., via a metatag) butalso define the start of a desired time index within the file that theuser wishes to define as the subject of the metatag. A second Swoteinput in the same or different angular direction may indicate the closeof the index. Alternately a second Swote input in a given angulardirection may further define desired/undesired times within the file.Different combinations of angular directions of Swote input and othercommands are contemplated. Some of the defined content may be analyzedfor sound, image, text to determine potential features in a video framethe user likes (e.g., an actress, her red dress etc., dialog beingspoken, soundtrack) and then this information may be integrated into theuser's profile/analyzed/supplemented with a taxonomy etc. Thisinformation may be analyzed to determine context and be integratedwithin an advertar.

In one embodiment, a plurality of commands (e.g., twelve commands) suchas Swote input up, copy, paste etc., could be assigned angular positionscorresponding to the numbers on an analog watch. At the twelve o'clockposition, if the user performed the Swote gesture in that direction, theSwote up input command could be displayed as being selected and upondeselecting the content, the could be activated, at the one o'clockposition, the paste command and so on.

In other embodiments, different commands can be input based on thedistance the brand/content item 104 is from the original input point 112by finger or stylus 106. For instance, a first command may be associatedwith Swote up input at X radial distance from point 212, then addingfunctionality of opening content 104 at X+1 radial distance and furtheradding functionality to send brand/content item 104 to “favorites” atX+2 radial distance. Alternately, the functionality may change entirelyat X+1 radial distance. Different combinations of the commands disclosedin this application are contemplated. Indicia indicating which commandsare selected may be displayed as well.

Swote functionality can be input on any content via a variety ofgestures. For example, in response to a double tap gesture on the topportion of an item of content, a Swote up input can be executed and inresponse to the same gesture in the bottom portion, execute a Swote downcommand can be executed. Another gesture to input a user's affinity forcontent is selecting content by placing two fingers on it and pinchingand zooming to increase/decrease affinity. In another embodiment, thecontent may be selected and a substantially circular “swirl” motion canbe made in a clockwise/counter clockwise direction to increase/decreaseaffinity.

In one embodiment, which is particularly useful on a touch screendevice, the final input may the input just before a gesture. Forinstance, entering the final input like that shown in FIGS. 1-4 may notfeel intuitive. In other words, upon selecting the desired command, theuser may not feel comfortable merely lifting her finger off of thescreen. Here, upon desired command selection, the final input may be theinput just before a gesture. Specifically, upon selection, instead ofjust lifting her finger off the screen, the user makes a swipe gestureup to the upper right and then lifts her finger off the screen. Thiscreates a more affirmative and intuitive input sequence for some users.

Data Collection GUI (Swote Command Zones)

Along with the above discussed Swote gesture movement in an angulardirection, a command zone embodiment is yet another embodiment of usingSwote input. FIG. 5 illustrates an embodiment 500 of the parsing of thespace around original input point 502 into a template with commandzones. Upon receiving initial user input by user 504, various zones canbe designated around said point 502 and/or brand/content item 104 byoverlaying a template. For instance, content 104 may be associated withfour different zones, each with a different functionality. In otherembodiments, any point on the display such as the center of the contentmay be the point around which a command zone template iscreated/displayed.

The template may be overlaid using initial point 502 on any such contentor any other desired point and different content may receive differenttemplates with different commands and/or different command size/shapedzones. For instance, news stories displayed on a mobile applicationrelated to a certain politician may be associated a first template and around picture of a movie frame may be associated a different templateeach with different commands, zones and sizes to accommodate thedifferent content such as a round template with “pie slice” shapedcommand zones.

Here, arbitrary command zones 506-516 are created by drawing lines518-522, through the corners of the content 104. Other methods ofcreating zones in any shape and number in relation to the content 104,initial input point 502 or otherwise are contemplated.

The functionality associated in each zone can be selected and/orexecuted by moving a cursor, finger, stylus etc. and/or brand/contentitem 104 into each zone. For instance, as brand/content item 104 issubstantially moved into zone 506, the zone's functionality is selected.As the user unselects content 104, the functionality of zone 506 isexecuted. In one embodiment, the functionality of zone 506 is to scrollleft in which the brand/content item 104 is scrolled left. Content thatis scrolled left/right and does not receive Swote up/down input may beviewed as “neutral” in one embodiment and weighted accordingly whenadded to the user's advertar. As the user selects the content, indiciaof the selected functionality may be displayed and when the userunselects the content, the left scroll is executed and optionally,indicia of the executed command is displayed.

In this embodiment, zone 508's functionality is to input Swote up inputwhich is a command to indicate the user's favorable view of thebrand/content zone 510's functionality is to both input Swote up inputand open the content (e.g., click on the content), zone 512'sfunctionality is to scroll right. Zone 514's functionality is to inputSwote down input which indicates a unfavorable view and additionally itselects the command to indicate that “you” already own or have viewedthe brand/content while finally, zone 516's functionality is to inputSwote down input.

In another embodiment, brands/content are shown like that in operation 1in FIG. 14. A user may select brand 1 which is illustrated in the toprow of the brand sorter, in which Swote command zones may appear. Theuser may select a command zone which executes a hyperlink which takesthe user to the brand's website while simultaneously moving the brand 1into the middle row of the brand sorter where brands 3, 5 7 and 8 areillustrated. This embodiment may also work with Swote angular directionembodiment.

Various shapes and sizes of command zones are contemplated. These can becustomized to the content e.g., diamond shaped content has a diamondshaped template overlay with a plurality of command zones. Each templatemay have any combination, shape, size and number of command zones.

In one embodiment, a command such as the user pressing the “shift” keymay display a menu of the available commands associated with a piece ofcontent. In a touch screen embodiment, touching and holding the contentfor a substantial period of time may trigger a display menu.

As discussed for the angular direction embodiment, content 104 can beassigned commands unique for that particular content such as type, thedata within that particular content (e.g., for a political story,commands to “vote” or for a celebrity dating story commands to “dump”etc.).

Swote Data Analysis

The analysis of content item 104 and related data may be conducted todetermine the “meaning” of the brand/content item 204 and/or the meaningof the user's gesture actions (e.g., Swote up input) as well as relatedinformation and associated statistical probabilities. “Meaning” may bedetermined in a variety of ways such as examining the data discussedabove as well as probable context of the input, taxonomies, user'shistory of inputs, browsing history, past profile information etc. Theoutput from this analysis may be new categories of products/services theuser is interested in and an associated statistical probability, a newupdated probability for a previous category concerning one or moredemographic/interest characteristics associated with the user, a newlink and associated probability between the user's advertar and anotheruser etc.

The analysis above may be via semantic tools, contextual tools,image/audio analysis tools, taxonomies, brand sorting, associating tomarketing data (e.g., statistical probabilities) social media trends andother methods.

Determining the “MEANING” of Swote Data

In one example, Britney Spear's husband appears in a picture thatreceives a Swote up input by the user. The picture or pointers to thepicture are analyzed locally or remotely. Here, characteristics andassociated probabilities from the Swote input may be added to the user'sinterest graph or Advertar.

On a local or remote computing device, the image and optionally, theassociated data such as the URL it came from, other files it is relatedto by tags, subject matter (other related Brittney pictures), other datait is stored with or related to by data links (other files/web pages itappears on), URLs it is related to, search engine results, other facesor features appearing in the picture (for sound files-other sounds, forvideo other features in the video, for text files other text or acombination of these) is analyzed as desired.

As above, the content itself may be mined for content. For instance, thecontent referenced by the pointer is analyzed and the husband's face isrecognized by facial recognition technology and identified as herhusband Kevin. In response, “Kevin”, “Britney Spear's husband”, “gossip”tags etc. are associated to that picture via taxonomy or marketing dataand associated statistical probabilities previously discussed in theincorporated patent applications. The URL and associated text in thenews story and captions where the picture were displayed to the user isalso associated to that picture and meaning (characteristics andprobabilities about/to the user) can be derived as well such as viasematic or other methods. The Swote input up command can be interpretedas the user liking this picture and weights and tags to this data mayreflect this. This data may also be added to the user's advertar.Content recommendations can be made based off the revised advertar as aresult of this self-profiling by content curation. For example, theadvertar may be offered an ad for a shirt with his face on it.

In one embodiment, a pointer to content item 104 such as a URL where thecontent is stored is received along with an indication of a “positive”Swote input. This data is also associated with the user's advertar. TheURL and content item is analyzed and it is determined the URL is from awebsite associated to celebrity gossip that is frequented by youngfemales who like the GAP™ brand. A date, time, place, mobile devicetype, operating system and other data on the user's device is recordedas well. Here, the date was on Valentine's Day at 8 pm on an iPhone fromher home. The meaning from this may be that the user is a single femalewho may be affluent and also likely interested in online datingservices. Various probabilities can be assigned given the totality ofthe gathered data.

Swote data analysis can also consider the context of the input via whatcommands are input and other factors—such as if the user owns it, wouldbuy it; send to friends, copy, delete it, paste, hate it, your profile(it was determined you are an enthusiast), other voter's, time, place,URL, merchant, color, speed of the Swote input (the user may be likelyhesitating during input).

In another embodiment, the user's picture that received a Swote up inputof Britney is analyzed and though image recognition technology, herdress is recognized as brand X and the background in the picture isidentified as Hawaii (either by image recognition technology, meta tagsattached to the picture, information from the webpage (captions) etc.).The “meaning” assigned to the picture may be new or modified statisticalprobabilities that the user likes Britney, Britney's dress, Britneywearing the dress, Hawaii, Britney wearing the dress in Hawaii, Britneyin Hawaii, Hawaii and the dress, Brand X and the dress and variouscombinations etc. This information may be entered into her advertar.

In another embodiment, a taxonomy is used to contribute to statisticalprobability characteristics within a profile/interest graph. Forinstance, if the interest graph had a characteristic of a beach scene Xbeing liked by people who are very “fashionable”, the above example mayincrease the probability that the user is fashionable. In addition, theBrand X dress in Hawaii may be interpreted as the user being interestedin tropical fashion which may influence the fashion characteristic suchas tropical fashion. In addition, the user's characteristic of “likes totravel” may be increased since the picture was in Hawaii. The user'sbrowser history or past purchases might also be examined to confirm aninterest in travel and to which destinations. Other goods/services frombrand X and related goods/services, other people who wear the dress,relevant news articles etc. and information related to Hawaii may berecommended to the user and/or influence her profile characteristics ina similar manner.

More granular determinations are possible with more detailed user input.For instance, a picture of Britney Spears with several of her friends issomething a user may input a Swote up input on. However in reality, theuser may only like Britney Spears and not her friends. The user mayhighlight or otherwise only select and input a Swote up input BritneySpears and not her friends. This may be accomplished by preprocessingthe image with facial recognition technology and allowing each face toreceive a Swote input up or down (or with whatever commands are desired)with optional indicia next to each face in the picture to indicate Swoteinput potential. A “thumbs up” tag is associated to the desired facesand a determination to refine her persona for only Britney Spears andoptionally purposely omit positive tags related to her friends. Inanother embodiment, the plurality of faces may be receive Swote input sothat the faces may be ranked relative to each other (most favoriteperson to least favorite person) and appropriate tags associated inresponse to each Swote input. In other embodiments, brands such aslogos, products and other objects could be recognized. Additionally eachtype of content may be given different commands such as a person mayreceive a command of “like her” vs. a product may be given functionalityof “already own it”. In another embodiment, content may be examined viapast user input, marketing data, social/interest graph, profiles and maybe assigned commands based upon historical trends. For instance, ifBritney was largely “hated” by a large segments of the users, thencommands and weights could be assigned like a large probability ofhating her or commands or “hate her” or “really hate her” etc.

For instance, in another example, a user inputs a Swote up input brandX. Analysis by a remote or local computing device is done by analyzingsimilar or related content as optionally determined by ataxonomy/marketing data. Then, demographics related from at least thatbrand as determined from marketing data, the URL the brand was chosenfrom (e.g. the brand was originally shown on an EBay or Amazon page) andthe URL can be analyzed for information (the URL was for a handbag etc.and related URLs) and associated to the user's profile with associatedstatistical probabilities. A search can be done for relevantproducts/services/news/reviews for the brand and recommended to the userbased on the profile or optionally only the information above.

In another embodiment, a user goes to a social media page and inputs aSwote up/down on other users, and data on their pages. For example, theuser inputs a Swote down gesture on a movie star Facebook profile, butinputs a Swote up gesture on a picture of her and her handbag from acertain movie. The content is analyzed and it is optionally concludedsince she Swoted down her Facebook page, but Swoted up the picture, itis determined that the user likes the handbag and/or the movie asopposed to the Facebook profile. Further questions/analysis/data sortingcan be done to determine if she likes one or both the handbag and movie.

In yet another example, a user can Swote on a picture objects can thenbe recognized and data can be associated to it via taxonomy, GPSlocation where the picture was taken, comments about the picture insocial media can create modifications to her profile.

In another embodiment, the user ranking/voting from Swoting can becombined with her friend's Swote input done by the same or similarmethods. This can be useful when determining common interests such asvia an interest graph. As such, friends or strangers with previouslyunknown common interests, geography etc. can be recommended ads (e.g.,deals which may require multiple people to fulfill or group vacationads) to people with similar interests determined at least in part by theabove methods.

Tag weighting may also consider past consumer actions such as previousSwote input, Swote input of associated users, purchase history from anassociated account, browser history, personal data such as demographics,brand Swote input, location history, associated devices, credit cardhistory etc. For instance, people who like red dresses may frequentlylike red shoes and other red objects. In turn red, objects associated tothe user's profile may receive additional tag weighting. This frequencymay justify the term red shoes being attached to the user's persona soshe may get ads for red shoes.

In one embodiment, tags, and probabilities associated to content mayreveal information about the content in relation to other users. Forinstance a picture rated disliked by a large number of users may reflectinformation about a user who likes the picture.

In one embodiment, content could be received without a tag. Tagging andweighting could be executed regardless based on historical trends, theuser's location, date, time of input etc.

In one embodiment, a brand such as “Subaru”™ is received with a Swote upindication. Marketing data pertaining to the brand could tag and assignweights to the content while considering the Swote up and to otherinformation in an advertar as well as the location, time, date of theuser input of Swote up as well as the context the Subaru brand wasSwoted up such as a car expert forum or the Consumer Reports™ website.

Multiple Swote Data Feeds and Other Swote Commands

FIG. 6 illustrates another embodiment of Swote input. Here, the contentis from multiple data feeds such as Facebook™ in which some examplefeeds are illustrated in FIG. 8. In addition, local files, emailaccounts, websites, EPG data and various other content sources such asblogs, news, TV show/movie data may be used.

As illustrated, feeds on a page can be sorted into different interestareas 600 such as work, personal, home, leisure etc. Sorting may be viamatching similar tags (as described above) from the content within feeds602 and 604 to the user profile, image similarity, taxonomy etc.

The user may input Swote input on each piece of content displayed. Forinstance, content 604 has been Swoted as indicated by indicia 606 asthumbs up. While content 602 is yet to be Swoted by the user. As theuser Swotes, in this embodiment, a thumbs up tag is associated withcontent 604. Tags and other information associated with 604 may beevaluated by previously discussed tools to determine meaning and contentscope may be expanded by a taxonomy.

FIG. 7 illustrates operations pertaining to linked accounts that may beexecuted when content 604 has been Swoted. Icons 608 and 614 illustrateexemplary feed/accounts that the user may be associated with. Theseaccounts can be the sources of the content 604. For instance, the usermay have active accounts that are linked to the operations of the Swoteinputs in the embodiment in FIG. 7 on Twitter™, EBay™, etc. The user mayselect various actions to happen on these linked accounts upon a Swote.

For instance, for a Picasa™ account, if the user inputs a Swote input upas indicated in column 612, the Swote operation may send information tothe user's Picasa account as indicated in column 610. In contrast forthe user's fig account, if the user inputs Swote input down in herStumpleUpon account, no action is taken for that account.

Augmenting Collaborative Filtering with Swote Input

FIG. 9 illustrates two webpages 900 using a Swote gesture on resultsfrom collaborative filtering. Here, Swote input enables the user aconvenient and intuitive method of entering information to supplementher Amazon™ product recommendations.

In this example, the bottom row of icons are collaborative filteringresults displayed to a user based upon previous input such as itemsbrowser/purchased etc. However, collaborative filtering as well as othersimilar techniques, while very popular with recommender systems has somesignificant shortcomings as opposed to methods discussed in this and therelated applications. Specifically, collaborative filtering assumes thatlarge groups act the same, but individuals may not act the same as agroup (e.g., atypical purchases of goods such as a small number of usersliking sardine ice cream). Collaborative filtering benefits if a userinputs their preferences which “pushes” or “pulls” the user to or awayfrom datasets associated recommended or less recommended products.

The tools illustrated in the top row, present a solution to theseproblems by providing a user with a convenient way to quickly andintuitively refine their collaborative filtering results. Here, Swotetools are used to let the user 904 refine the recommendations made withcollaborative filtering or other recommendation technologies. In thisembodiment, user feedback is gathered to supplement theserecommendations by ranking the recommendations via either the abovediscussed angular direction or command zone Swote embodiments.

Element 902 in the bottom row is illustrated as a recommendation to theuser as determined from collaborative filtering. In some existinginterfaces, Amazon requires the user to click on the recommendation andasks for a rating 1-5 stars as well as asking if the user already ownsit and if she is interested in it at all. Each of these questions isanswered separately with associated separate inputs. The commands inFIG. 5 allow a user to answer these questions with a single compoundcontent gesture.

Performing a Swote input for the element 902 solves these problems byletting the user simply move the icon up with her finger or cursor.Granular affinity is illustrated as a +2 rating is given in the top row.Negative granular affinity can be given as illustrated in the bottom andtop rows where recommendation 906 is moved by the user into a −3 ratingby the user moving the recommendation down. Neutral ratings can be wherethe user simply leaves the recommendation unchanged.

Various methods of ranking can be done by inputting Swote input on theicons e.g., moving the icon up, down, left, right, speech, gesture inputetc. The interface in FIG. 5 may also be used in which the user maySwote in different angular directions in addition to up/down to indicateownership and interest as well as a positive/negative rating.

The input from the user's various Swote gestures can then be used tosupplement the collaborative filtering engines. In order to enable amore customized persona/recommendations/offers when the results from theinput of Swote gestures are incorporated into the collaborativefiltering engines.

In one embodiment, a Swote input may be enabled without and the resultsincorporated into the user's profile to enable greater accuracy ofcollaborative filtering. This process may be repeated as necessary. Thismay be executed whether the content has been tagged or otherwiseassociated with information or not. If information was associated, thetagging and statistical probability analysis via Swote input may alsoconsider this information.

In Game Ads

FIG. 10 illustrates Persona Driven In-Game Ads 1000. Image 1000illustrates a frame of a video game 1004. As illustrated below theimage, a user's advertar (icon represented by the advertar on thebicycle) 1002 is combined with the user's characteristics in andparticipation in the “need for speed” video game 1004 on a computingdevice such as audience engine 1006.

In one embodiment, the communicating device may locally or remotelyassociate the user with an advertar. The computer on which the videogame is running is programmed to consider the demographiccharacteristics of the user as requested by their advertar along withrelated context (e.g., cars, sports, likely demographic of the players,other connected players etc.), and determine content to display in thegame such as ads which may shown on the cars or other places in thegame. When the user is playing the game, ads 1008 and 1010 may appearwhich were determined in the last step. Thus, instead of unsightlypop-up ads, ads in this embodiment appear on locations normallyassociated with ads such as on race cars, billboards etc. This exposesthe user to content such as advertisements with minimal invasion in amore “natural” and less intrusive setting for the ads.

Advertar information collected by various methods including Swote inputand the previously discussed brand sorting is sent to the audienceengine for processing. This information, new ads etc. can be determinedand displayed to the user as illustrated in 1008.

In another embodiment, these tools can be used outside of video games.For instance, on game console, IPTV, media player interface screens,“ticker” bars that scroll information across a screen.

In other embodiments, the user's device is programmed to determine thetype of content the user is watching such as a cooking show and displayads related to the particular cooking show and other factors asdetermined from the demographic characteristics repeatedly by theadvertar 1002. The display of the ad could happen in place oftraditional commercials, on ticker screens, after the program has ended,or on other devices as discussed in the incorporated patent applicationsetc.

In some embodiments, information stored in the user's device such as thegame console, the user's game playing style (e.g., typically moreaggressive players are younger males), game character customization suchas the type of appearance/clothing the user picks for his/her charactercan be added to the advertar 1002.

Ad placement in a game or other content can be auctioned in real timebased on viewership (e.g., number of people, number of people withcertain characteristics/advertars etc.).

Dynamic Voting Adjustment Tools

Dynamic tools as disclosed are helpful in preventing a single or smallnumber of “outlying” inputs from unduly changing a profile. In oneembodiment, the contribution of potentially “outlying” inputs isminimized through weighting. For instance if the user's profile includedthe category “spa deals” and the user inputs a single data point ofinterest for this category (the user is actually not interested in spadeals), the input may contribute a relatively small weighting to thatcategory within the user's profile. Thus, one or a small number of adsmay be shown to the user to confirm her interest.

However a relatively large number of inputs will weight the “spa”category in her profile to be more significant and thus show ads to her,induce a taxonomy analysis to find out related areas etc. In anotherembodiment, a single or small number of positive/negative inputsregarding a “spa deal” in a particular context (such as if the reviewerhad been to the spa or the input was received from the spa locatione.g., via her smart phone) may cause the single input to be weightedmore than a single input.

For instance, it if was determined if the user was at a spa (via herdevice's GPS), had lots of spa related URLs in her browsing history, hadpreviously purchased spa deals etc., the spa input could be weighted topromote/demote the category more than a single input.

Adjusting a Profile via Contextual Input

Input into an advertar can occur in a variety of ways in addition toperforming a Swote gesture. In one embodiment, a computing device isprogrammed to analyze and thus add information (via the analysis toolsdiscussed for Swote input) to a profile from pictures a user has taken,sounds recorded, Shazam™ inputs, email, Facebook™ and reviews such asYelp reviews. For instance, a picture of a flannel shirt is taken by auser at a geolocation recorded by the device. She then sends the picturewith an email to a friend using terms like “awesome” and “would match myother brand of flannel pants.” Analysis by a computing device can deducecontextual information that she likes the particular shirt, she ownsother flannel clothes, or other information can be gathered about herfriend from her email address or account information, relevantinformation of the location (e.g., close to other stores that may givecontext), as well as objects that can be determined in a picture whichmay infer that one likes, dislikes the objects given previousinformation gathered from the user e.g., contextual information inprevious pictures, positive reviews (Yelp™ reviews) purchases and soforth. In another example scanning a QR code at a certain physicallocation can serve as contextual input. For instance, scanning a QRcode, input a swote up gesture while at store X, may indicate that youlike the particular store, brand, location etc.

Miscellaneous Tools

Feedback Based Ad Bidding system: Given the ability for a user to enterinformation such as feedback via the Swote gesture and the Brand Sorter,a bidding system can be implemented to monetize this enhanced consumerinformation. This may be applicable to advertisers, contentsellers/makers and the like.

Previously, search engines ranked displayed search results via:position=bid*relevance or a similar type formula. Given the enhancedconsumer information, the following formula may be used to monetizeconsumer data: position=bid*consumer feedback*Personal Interest Networkvalue. In one example, consumer feedback maybe from the consumervoting/Swote input on a plurality of devices, or a plurality ofdifferent consumers. A Personal Internet Network value may be adetermined by quantifying preexisting data described in thisapplication, brand sorting and/or a combination in addition topresenting a consumer with checklists etc. For instance, a PersonalInterest Network value for a user may be higher if she has asubstantially large amount of profile information. The resultingincrease of relevance to the consumer results in a correspondentlyhigher charge to the advertiser.

Primer on Interest Graph/Profiles

As discussed in the incorporated patent applications, advertars/profilesof a user may reflect characteristics and associated probabilities amongother information. As such, interest graphs may be a part of a profile.As the user inputs information by sorting content and/or performingSwote input (see below), advertars may be created or supplemented withthis data.

In one embodiment, an interest graph refers to the specific and variedinterests that form one's personal identity, and the attempt to connectpeople based on those interests. Individually, this may mean differentthings one person is interested in—be it jogging, celebrity gossip, oranimal rights and other interests that make up their likes and dislikes,and what has more meaning to them over another. On a broader scale, it'sthe way those interests form unspoken relationships with others whoshare them to create a network of like-minded people.

As an example, FIG. 15 illustrates one embodiment 1500 of an interestgraph. As illustrated, are a first user's advertar 1502, a second useradvertar 1504 and a stranger's advertar 1506 to whom a user has nopersonal connection to and may not even know. Link 1508 is a statisticalprobability for a characteristic for an advertar as well as arelationship probability between the interests it connects, while 1510is a interest in “cooking for singles” in which stranger 1506 and user'sadvertar 1502 have in common.

As opposed to a social graph (which may also be included or maycontribute to an advertar) an interest graph focuses on shared interestsregardless of personal connections (such as the “cooking for singles”interest) while a social graph focuses on connections based on personalconnections. (In some embodiments, advertars may incorporate socialgraphs as well or just social graphs alone).

While the social graph consists of a users known to other users, theinterest graph consists of what they like, what moves them, and thefacets of their personality that, in part, make up who they are andoptionally users they may know. These connections can be much stronger,and much more telling, than simply who they are friends or acquaintanceswith. Two people being linked together because they knew each other inelementary school or work at the same job doesn't necessarily indicateanything about them beyond their connection to each other. And for thepeople involved, it doesn't always foster a very strong or lastingconnection. As such, an interest graph may offer more insight into eachperson's personal tastes, preferences and behaviors.

Thus, given X users connected in an interest graph who share commoninterests, are most likely more interested in the same advertisingcompared to users who do not share these users common interests. Inaddition, characteristics and associated characteristics (e.g., via ataxonomy) as well as statistical probabilities of those users can bestudied and offers, interests, products and other goods/services can bedeveloped specifically for those demographics. This provides a highlypersonalized experience and also connects a user to users who havecharacteristics in common who otherwise might never meet each other.

Rapid Information Presentation Tools

Another problem faced by consumers is how to redeem coupons, offers,vouchers, offers, token, deals, discounts, rebates, reward (hereincoupons) when purchasing goods. Previously, retailers using scanners andother input devices have discouraged or even forbidden the use of mobiledevices (e.g., smart phones used by consumers) for information inputsuch as coupon barcodes and QR codes. This is because it is very timeintensive for a consumer to scan a barcode, then “flick” or otherwisedisplay another barcode to the scanner. This often awkward and timeconsuming process results in long lines as a consumer “fiddles” with thedevice during this process.

Related to this problem is another problem that has historically plaguedinformation presentation at retailers and other entities namely the needto minimize the implementation cost of new methods of informationpresentation. To address these problems the disclosed technologyprovides tools for quickly presenting information without significant orideally any change to the POS (Point of Sale) equipment such as cashregisters, scanners etc.

Disclosed are automated Rapid Information Presentation Tools. Thesetools provide a method of automatically presenting information to aninput device in a fast, automated way, which is convenient and will savetime. Specifically, these tools may minimize user interaction duringpresentation which will prevent user error and time spent “fiddling”with their computing devices.

FIG. 11 illustrates an embodiment of said tools. In the exampleillustrated, a consumer's computing device rapidly presents coupons at apoint of sale scanner. In operation 1, a computing device 1102, whichmay be a phone, smart phone, laptop, watch, MP3 player, tablet, netbook,phablet, ultra book, e-reader etc. is presented to an information reader(fixed or handheld) such as scanner 1106 which illustrates both fixedand handheld versions. Here the scanner 1106 is a barcode scanner. Thescanner reads coupon information displayed on the computing device thatcan be in the form of an SKU, barcode, a QR code, a picture of a coupontaken a user, an electronically clipped coupon or any other visual,audio, tactile, vibration, wireless, NFC information etc. in which anappropriate scanner may be used.

The scanner 1106 reads the information from a mobile device 1102 as acoupon 1104 is displayed on the device's screen. In this embodiment, theinformation reader is an infrared barcode scanner while in otherembodiments; the reader can be a QR code reader, an optical camera, orother information input device. The device 1102 can any type ofcomputing device as explain elsewhere such as a iPhone 5™, SamsungGalaxyS3™.

Once the information from the mobile device is received by the reader,then in operation #2, the scanner gives confirmation of input such as anaudible “beep” 1108. In one embodiment, the user's device is programmedto give a confirmation indication such as a camera flash, screenindication, beep or vibration upon each successful entry. The outputfrom either the scanner and/or device could be any visual, audio orother output such as a electronic signal, vibration etc.

In operation #3, in response to the mobile device detecting the “beep”1108 or other confirmation of a successful information input, via amicrophone, the mobile device is programmed to then automaticallypresent another piece of information (e.g., another barcode 1110) to theinformation reader, for quick and automatic input without the userhaving to do any further action.

This process may repeat for each coupon or item of information to bepresented. When all the user's information is presented, then the mobiledevice can indicate to the user through visual, audio tactile feedbacketc. that the information has been presented and received. The mobiledevice might also signal to the information reader that the process iscomplete and the information reader is programmed could give a signal tothe user that the process is complete. In the same manner, if there wasan error scanning (problem with information input or invalid couponetc.), the mobile device or scanner could indicate to the user of aproblem by displaying the problem on the device or the scanner itselfcould indicate a problem. An embodiment of this process is illustratedin operations 1124 which is stored in memory 1204. Device 1102 may beconnected via network 1310 to the internet, any network, coupon site,offer networks, ad, networks, brands, advertiser websites and otherusers/devices 1124. In addition, it may be connected to audience engine1122 which may record device 1102 activity such as coupon redemption inan advertar.

Embodiments

In one embodiment, the operations described above occur in a grocerystore checkout line. The user presents the device over the scanner witha first barcode on the screen or has the grocery clerk point a handheldscanner toward the user's mobile device. The software on the device isconfigured to detect a “beep” or other input from the input device whenthe barcode is read by the input device. Upon successful scanning of acoupon barcode, the scanner beeps which instructs the mobile device (viathe microphone) to present another barcode for scanning. Upon completionof the scanning of the desired SKUs, the device may trigger a signalthat the process is finished so the user may remove the device from thescanner. Said signal may be a camera flash, vibration, haptic feedback,sound etc.

In one embodiment, the mobile device is programmed to present a newbarcode upon a “beep” of only the scanner it is presenting to instead ofother nearby scanners emitting beeps near the mobile device but engagingin transactions with other users (scanners in other grocery storecheck-out lines). This maybe accomplished by presenting a new barcodeupon hearing “beeps” of only a certain volume, proximity or other methodthat limits input only to the relevant scanner.

In one embodiment, the user's device may be placed in a substantiallyflat position against the scanner surface. In this embodiment, thescanner is integrated into the cashier counter so the surface issubstantially parallel to the ground. Upon the device's detection via asensor such as a gyroscope which verifies that the device issubstantially flat against the scanner, and optionally, detection of ascanner signal by the device's camera such as the infrared scanningbeam, the device may start displaying barcodes and other information.Optionally, a timer delay which delays presentation until after thedevice detects it is substantially flat against the scanner to theground and/or, detects the scanning beam to minimize error.

In another embodiment in which the user does not want or cannot use asubstantially flat scanner, the user may start a timer in which she maydelay SKU presentation for several seconds to minimize error until thedevice is presented in front of the scanner. Alternately, upon detectionof the signal and optional detection of confirmation of input (beep fromthe scanner etc.), a new SKU is display and optionally, feedback fromthe phone such as on the display screen, beep, flash etc. is given toeither/or the cashier and user.

In another embodiment, the presentation may begin each time a scannersignal is detected by the camera or when the scanner emits a beep whentransmitting a scanner beam. For instance, a hand scanner may transmit ascanning beam that is directed in the general direction of the device'sscreen. The device is programmed to use the camera to detect the signaland then present a coupon. Upon continuous or reintroduction of ascanning signal, a coupon may be presented. A button or other inputmechanism may be used on the device screen for the user to keep thecoupon on the screen in the case of a misread. A “rewind” button mayalso appear so the user may go back and attempt another read of thecoupon.

In one embodiment, the first barcode that is displayed is the consumer'sloyalty card barcode. In yet another embodiment, when done presentinginformation, the device will display a neutral screen e.g., black screenor text saying done to prevent the final barcode from being repeatedlyscanned.

Assembling the Coupons

There are a variety of tools in which a user may assemble the data forpresentation. Said data may include information to be presented at apoint of sale such as brands/content/ads including: coupons, SKUs, ads,offers, discounts, vouchers, QR codes, referrals, sales, loyaltyaccounts (e.g., loyalty barcodes) etc.

In one embodiment, the user may take physical ads/offers/coupons andtake pictures of them with her device. The device may either recognizethe barcode/QR, image or other information code on the coupon, analyzethe image and determine what is in the image and then associate theimage to one or more ads/offers/coupons or other information to bepresented at the POS.

Once the coupon is recognized, a representation of the coupon may beshown to the consumer such as the picture or other image/text. The usermay then categorize the coupon into different stores, categories;discount levels, to be shared with friends, a folder indicating wherethe coupon was from (e.g., which periodical). Categorization may be viadrag and drop mechanism as discussed in previous patent applicationssuch as via tools similar to the previously discussed photo sorterembodiment.

In other embodiment, the coupon itself can indicate whichstore/manufacture/category it is from/for and where it can be redeemedor stored or where it should remind the consumer it should be redeemed.That may be done by SKU(s), QR code, text (the device can use OpticalCharacter Recognition to read the text and associate the validstore/manufacturers). The device may tag the coupon with any relevantinformation such as the name of the store offering the discount, expirydate and any other terms or conditions to a picture of a recognizedcoupon.

In one embodiment, an image of a coupon is enhanced to promote betterpresentation to the POS, or the SKU or other information is representedin a new, cleaner presentation to mitigate errors during presentation.For instance, a “grainy” picture of a coupon could be associated or bereplaced by a newly constructed barcode which would not have any errorsintroduced by wrinkled paper, a poor camera, bad lighting etc.

In another embodiment, once the coupon is recognized, the device mayrefer to a remote database to get more information. In one example, thedevice recognizes a coupon and transmits that information with theuser's current GPS coordinates and/or a history of which stores shefrequents. The remote server then sends back related products/storeswhich are in the area or are on routes she has taken with her device.

In another embodiment, the user may gather coupons electronically. Inone embodiment, the user may take screen shots of coupons or otherwise“clip” the coupons electronically and use software to recognize thecoupons, SKUs, objects within the coupon. The user may store the couponson an account at a remote server, or a local device.

For instance, the user may have an account or other otherwise create adestination for the coupons. In one embodiment, the user mayelectronically “clip” a coupon in which the coupon may be associatedwith other electronic and non-electronic coupons. Once the user hasfinished clipping such as via a web browser in which she selects whichcoupons are clipped, the clipped coupons are associated with an account(e.g., loyalty account) and/or sent/electronically printed to a device.

For instance, after a user has electronically clipped coupons on atablet or other device, the coupons are sent to a smartphone via WIFI orother tools. An application on the phone receives the clipped coupons orotherwise remote and login credentials in which the coupons are storedat.

In one embodiment, the device may be associated with an email, otheridentifying information, device or software ID or loyalty programaccount such as a Safeway™ Rewards Card. The clipped coupons are thensent to a mobile device for presentation at the point of sale. Inaddition, the clipped coupons may be analyzed and appropriate discountsoffered or new coupons may be offered based on the clipped coupons. Theclipped coupons may be automatically categorized by type, section of thestore (e.g. fruits/vegetables), expiration date etc.

Reminders may be sent to the user of expiration dates, as the dates getcloser. Reminders may also be based on geolocation to the store, or oncein the store, to the specific sections of the store where the goodsassociated to the clipped coupons are displayed.

A log/history may be associated to a particular account/device. This maybe used to determine which coupons were presented successfully and canreport the success/failure of a coupon to other users. For instance,certain merchants may not accept a coupon while others will. Thespecific result may be shared with other users to aid in redemption.

An app on the device may program the device to activate the camera andthen all pictures taken through the camera or send them for remoteanalysis. The images of coupons or images that are related to products,services can be recognized and trigger the display of a barcode, or QRcode to create a new deal or combination of deals based on where thescan is taken, from which magazine/newspaper scan is taken, items boughtso far etc.

In one embodiment, pictures the user took of a magazine illustrationwhich included a surfboard and a coupon for surfboard lessons isanalyzed. The coupon is recognized and the barcode is stored in thedevice for presentation later. The surfboard is recognized and a requestis made to a remote server for surfboard and related product/servicecoupons. The resulting SKUs are then received, stored and displayed tothe user on the device or a connected device.

Redemption Aiding Tools

In one embodiment, a device is programmed to track which couponspresented are recognized and thus can verify the running total to ensureaccuracy of the purchase.

For instance, the plurality of coupons presented may represent a certainvalue saved. In one embodiment one coupon may be worth $0.25 whileanother $0.75. Upon successful entry of the coupons as verified by a“beep”, the total saved should be $1.00. This amount can be displayed onthe device after successful entry of the coupons into the scanner. Thisnumber should correlate with the receipts and POS display, which oftenlists the amount saved. The consumer can verify this and upondiscrepancy, discuss the issue with the merchant before leaving thestore.

In another embodiment, before presentation at the point of sale, a usercan see how much savings her coupons will enable. This estimate ofsavings has many applications. In one embodiment, as the user shopsaround the store or even before entering the store, she can enterproducts and/or costs manually, or by scanning barcodes, taking picturesand using image recognition, NFC or other methods enter prices ofproducts. The coupon savings are added to the total to give notice ofwhat she will need to pay.

In another embodiment, the coupon savings may be estimated at differentmerchants. The consumer can see which of her coupons are sold at variousstores and can see if she used the coupons at a given store, how muchshe would save over other stores.

Here, the coupons can be analyzed by merchants in her physical area. Theparticular products associated to each coupon can be determined if theyare offered/are in stock and for what price. The prices she will pay ateach merchant can be displayed to the user. The user will thus, be ableto compare which merchant in her area is offering the lower combinedprice and if it is even in stock.

Micro-wireless devices in the store indicating different sections of thestore or different products can be used to help route the consumerthrough the store in order to find the products associated to thecoupons she has clipped. The various sections/products can be displayedon the user device overlaid on a map of the store and/or directions canbe given to the consumer. In addition, the relative directions of theproducts/sections can be displayed when the consumer looks through thedevice camera. The display may indicate what particular product/sectionis in a relative direction, what the distance to the destination is,products/sections related to the coupons or her profile etc.

In one embodiment, the action of a consumer presenting the coupon to ascanner and/or the successful input into the scanner may be recorded.This information may be incorporated into a profile associated with thatconsumer. In a manner similar to the Swote subject matter discussedabove, a favorable affinity tag may be associated with the coupon, thecoupon's expiry date, manufacturer, the product, the store at which thecoupon is redeemed, the price etc.

Combination Embodiments

As discussed above, clipped coupons may be added to a user profile. Thecombination of coupons a user clips/actually presents at a POS may beanalyzed and trigger new discounts in response to the clipped coupons.In addition, this combination in addition with a user's profile may alsotrigger discounts.

For instance, in one embodiment, a user is presented with a plurality ofcoupons on a computing device. In response to a user selecting coupon Aand B, a discount C is offered as a result of these selections. Thediscount C may or may not be advertised to users who have not selectedcoupons A and B.

In another embodiment, a user selects coupon A. The selection isanalyzed with the user's profile on a remote or on the local device.Given the user's profile, which may include shopping history, brandsorting information (as discussed in previous patent applications),geolocation information, browsing history etc., a further discount isgiven. For instance if her profile indicates that coupon A is from acategory of goods that the user has not purchased before but, asdetermined from brand sorting, is statistically likely to have anaffinity for, a larger than average discount can be given to incentivizea purchase.

In another embodiment, the user's selection of coupons A and B and theuser's profile history is analyzed and used to determine that the usermay like another product D. An incentive such as a coupon can be givenfor product D that may or may not be given or advertised to other userssince it is determined that it is statically likely from her profileand/or the combination of coupons A and B that product D is needed/oftenused or often desired with the goods associated to coupons A and B. Thisdiscount could also consider a specific store's inventory. For instance,if a significant excess of product D is in inventory, then a discount ora larger than planned discount could be offered to the user.

In yet another embodiment, the user's profile is analyzed once one ormore coupons are clipped. From analysis of the user's profile, adiscount of one or more of the clipped coupons or goods not selected bythe user may be given to the user. For instance, if a user is determinedto be statistically unlikely to be able to afford the good now, butmight in the future (e.g., a user who is a medical student), a discountcould be given to expose the user to the good in the hope of the userreturning to buy the good when she is in a better economic situation.

In another embodiment profiles from a plurality of loyalty accounts canbe analyzed with various information such location date on the user'sdevice, browsing history, purchase history, friend's data etc.Agreements between loyalty programs can be used as above to analyze andincentivize purchase of goods based on user's profile and clippings.

In one embodiment, data comprising profile data, purchase history, brandsorting, friend's data can be stored locally on the user's device andmay be shared with other devices. In this manner, a user who has atablet and phone can have access to the same data with either device.This may be accomplished by direct sharing or centralized storage andsharing on a remote device over a network.

The profile/coupon analysis can be done by calculating the vectordistances between persona vectors, the coupon vector as discussed inprevious patent applications.

Sharing Embodiments

The user's clippings and/or profile information may be shared betweenuser's. This promotes the clippings and incentivizes further purchase byother users. Characteristics of a profile may also be used to help matchcoupons from a user to another user.

A coupon clipping log may be kept in which clippings are stored andgeofencing reminders and/or coupon expiry date reminders will inform auser to use the coupons at the appropriate time/location.

Error Mitigation Embodiments

Reading errors may occur during coupon input. Various methods areavailable to prevent/mitigate reading errors.

Putting the device substantially flat against the scanner minimizesglare and other interference. To this end, the device can startpresenting SKUs to the scanner, once the device detects (e.g., viagyroscope) that it is substantially flat on the scanner. Intervalsbetween beeps can be calculated starting at a given time after thedevice starts presenting information.

When using NFC, a user can place the back of the device against thereader. The screen facing the user may display the status of whatinformation is being presented if reading is successful, redemption issuccessful (e.g., the coupon is not expired), keep a running total ofmoney saved etc.

If a coupon is not successfully read by the scanner as determined by alack of scanner beep, a predetermined time interval etc., the presentingdevice can skip the coupon and move on to the next coupon. Indication ofan unsuccessful read may be done upon a “unsuccessful beep sound”, asequence of the presenting device's flash etc. A status screen may beshown to the consumer/cashier with the coupon that was unsuccessfullypresented and a SKU to be manually typed in to the point of sale, areason why the coupon was not successful (e.g., expired, unreadable, notvalid at this location).

In yet another embodiment, a plastic, fabric or other material can beconstructed to form a covering over the scanner and the devicepresenting the information. The covering forms a substantiallylight-free environment, in which serves to 1) prevent glare from otherlight sources which prevents reading information off the device'sscreen; and 2) will serve as an optional trigger to the device.Regarding the second aspect, the device's light meter/camera may detecta lack of any substantial light after the presentation app is started asan additional/optional indication to start presentation of the coupons.For instance, the user may request the app start presenting coupons inwhich a screen with relatively low light emission such as a black screenwith font colors that emit relatively low light are displayed saying“please place on scanner to redeem your X number of coupons”. Thecashier may place the covering over the device and scanner. The devicedetects a substantial drop in light and begins presentation of thecoupons. The cash register may record the number of coupons and when theX number of beeps or X number of coupons are redeemed on the cashregister window, the covering is removed. Alternately, the phone mayring when the redemption is over.

Other Embodiments

Coupon clipping can be a reliable reflection about the demographics of aconsumer. As discussed, in the above previous patent applications,consumer selecting information about themselves such as would buy, wouldbe interested in a coupon about, is interested in a discount can bevaluable. The brands associated with their interest can be analyzed asdiscussed in said patent applications such as summing tags associated tobrands and forming a profile/persona vector.

When a consumer clips coupons, it is reasonable to infer that she isinterested to a relatively high level of degree in the coupons andperhaps the details in the coupons. Thus, a profile may be supplementedor even constructed from coupon clippings. Specifically, as the consumerclips coupons, the product and associated brand can be analyzed such asvia a taxonomy and tags associated to them. Alternatively, tags may beassigned directly by the entity that offered by the coupons.

“Brands” as defined in the incorporated patent applications as well ascoupons, ads, products etc. may be matched to said persona vector bycalculating the distance or other methods. Said brands can be presentedto the user as suggestions to purchase etc., or can be presented to theuser for brand sorting as discussed in the incorporated patentapplications.

In the embodiment of coupons, several pieces of information on thecoupon can be integrated into the profile and/or into an existingloyalty card account. For instance, the expiry date, percentagediscount, amount of discount, redemption locations, other redemptionconditions such as “must be purchased with X”, languages the coupon isoffered in (e.g., English & Spanish), retailer offering the coupon,specific product and associated manufacturer, combination of couponsredeemed, specific location/date/time the coupons was redeemed at etc.

These characteristics can be added to her profile and further brands,coupons, user specific discounts etc., can be offered to the consumer.

Integration with Loyalty Cards, Scanner Specific Details and Activationof Functionality

Integration of platform specific equipment specifications such asspecific input/output sounds and other properties/requirements forcompatibility by a retailer may be integrated e.g., Bad “beeps” maybeplatform specific.

In, one embodiment, the device itself can be triggered to activate theapplication or invoke functionality as the operating system levelthrough a variety of methods. For instance, location of the device incertain GPS coordinates or the detection of certain wireless signalssuch as from a retailers wireless network may invoke functionality.Input of a scanner signal as detected from a camera, NFC, link from atext, email or other signal such may trigger activation illustrated inoperations #1-#3 automatically.

In another embodiment, at the end of coupon assembly, a barcode or RQcode can be created to convey information about how, from where, thecoupon originator and other information about how the coupons wherecollected. This barcode may be read at the point of sale in addition topresented coupons/loyalty card to give extra information to theoriginator of the coupon. The information can include information forrouting/transmission to the originator or manufacturer.

In another embodiment, an app can be configured to detect by GPS whichparticular store the consumer's device is located at and upon checkout,present that store's particular loyalty card barcode. This isparticularly helpful if a coupon is valid at a plurality of stores eachwith different loyalty cards.

Common Material Background for Swote and Coupon Presenter TechnicalProblems Solved

As discussed in this document, the discussed subject matter solvesseveral technical problems. Specifically solved is the input andprocessing of user input information such as user affinity toefficiently determine user characteristics from content as well as novelinput of commands such as copy/paste on a small mobile device screenamong other computing devices. The related processing by a client/serveris also made more efficient due to the enhanced gathering ofinformation. Also solved is the problem of the user being overwhelmedwith irrelevant advertising. The advertar solution as discussed herein,creates efficiencies as the user can more easily filter information andtherefore be fed relevant ads.

Also solved is the problem of the need for rapid presentation ofinformation such as barcodes at barcode scanners. Specifically solved isthe input with a minimum of user interaction of a plurality ofinformation to scanners quickly and efficiently.

Description of Computer Hardware

Embodiments of the subject matter and the operations described in thisspecification can be implemented in digital electronic circuitry, or incomputer software, firmware, or hardware, including the structuresdisclosed in this specification and their structural equivalents, or incombinations of one or more of them. Embodiments of the subject matterdescribed in this specification can be implemented as one or morecomputer programs, i.e., one or more modules of computer programinstructions, encoded on computer storage medium for execution by, or tocontrol the operation of, data processing apparatus.

A non-transitory, computer storage medium can be, or can be included in,a computer-readable storage device, a computer-readable storagesubstrate, a random or serial access memory array or device, or acombination of one or more of them. Moreover, while a computer storagemedium is not a propagated signal, a computer storage medium can be asource or destination of computer program instructions encoded in anartificially-generated propagated signal. The computer storage mediumalso can be, or can be included in, one or more separate physicalcomponents or media (e.g., multiple CDs, disks, or other storagedevices). The operations described in this specification can beimplemented as operations performed by a data processing device usingdata stored on one or more computer-readable storage devices or receivedfrom other sources. A representative data processing device is shown inFIG. 12.

The data processing device includes “processor electronics” thatencompasses all kinds of apparatus, devices, and machines for processingdata, including by way of example a programmable microprocessor 1202, acomputer, a system on a chip, or multiple ones, or combinations, of theforegoing. The apparatus can include special purpose logic circuitry,e.g., an FPGA (field programmable gate array) or an ASIC(application-specific integrated circuit). The apparatus also caninclude, in addition to hardware, code that creates an executionenvironment for the computer program in question, e.g., code thatconstitutes processor firmware, a protocol stack, a database managementsystem, an operating system, a cross-platform runtime environment, avirtual machine, or a combination of one or more of them. The apparatusand execution environment can realize various different computing modelinfrastructures, such as web services, distributed computing and gridcomputing infrastructures.

A computer program (also known as a program, software, softwareapplication, script, or code) can be written in any form of programminglanguage, including compiled or interpreted languages, declarative orprocedural languages, and it can be deployed in any form, including as astand-alone program or as a module, component, subroutine, object, orother unit suitable for use in a computing environment. A computerprogram may, but need not, correspond to a file in a file system. Aprogram can be stored in a portion of a file that holds other programsor data (e.g., one or more scripts stored in a markup languagedocument), in a single file dedicated to the program in question, or inmultiple coordinated files (e.g., files that store one or more modules,sub-programs, or portions of code). A computer program can be deployedto be executed on one computer or on multiple computers that are locatedat one site or distributed across multiple sites and interconnected by acommunication network.

The processes and logic flows described in this specification can beperformed by one or more programmable processors executing one or morecomputer programs to perform actions by operating on input data andgenerating output. The processes and logic flows can also be performedby, and apparatus can also be implemented as, special purpose logiccircuitry, e.g., an FPGA (field programmable gate array) or an ASIC(application-specific integrated circuit).

Processors suitable for the execution of a computer program include, byway of example, both general and special purpose microprocessors, andany one or more processors of any kind of digital computer. Generally, aprocessor will receive instructions and data from a read-only memory ora random access memory or both. The essential elements of a computer area processor for performing actions in accordance with instructions andone or more memory devices for storing instructions and data. Generally,a computer will also include, or be operatively coupled to receive datafrom or transfer data to, or both, one or more mass storage devices 1204for storing data, e.g., flash memory, magnetic disks, magneto-opticaldisks, or optical disks. However, a computer need not have such devices.Moreover, a computing device can be embedded in another device, e.g., amobile telephone (“smart phone”), a personal digital assistant (PDA), amobile audio or video player, a handheld or fixed game console (e.g.Xbox 360), a Global Positioning System (GPS) receiver, or a portablestorage device (e.g., a universal serial bus (USB) flash drive), to namejust a few. Devices suitable for storing computer program instructionsand data include all forms of volatile or non-volatile memory, media andmemory devices, including by way of example semiconductor memorydevices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks,e.g., internal hard disks or removable disks; magneto-optical disks; andCD-ROM and DVD-ROM disks. The processor and the memory can besupplemented by, or incorporated in, special purpose logic circuitry.

To provide for interaction with a user, embodiments of the subjectmatter described in this specification can be implemented on a computerhaving a display device 1208, e.g., an LCD (liquid crystal display), LED(light emitting diode), or OLED (organic light emitting diode) monitor,for displaying information to the user and an input device 1206 such asa keyboard and a pointing device, e.g., a mouse or a trackball, trackpad etc. camera (e.g., optical, 3D and/or IR), proximity detector, bywhich the user can provide input to the computer. In someimplementations, a touch screen can be used to display information andto receive input from a user. Other kinds of devices can be used toprovide for interaction with a user as well; for example, feedbackprovided to the user can be any form of sensory feedback, e.g., visualfeedback, auditory feedback, or tactile feedback; and input from theuser can be received in any form, including acoustic, speech, or tactileinput. In addition, a computer can interact with a user by sendingdocuments to and receiving documents from a device that is used by theuser; for example, by sending web pages to a web browser on a user'sclient device in response to requests received from the web browser. Thedata processing apparatus 1200 may also include a wireless transceiver1212 such a cellular radio, WiFi or WiMax transceiver, Bluetoothtransceiver and a network connection 1214 etc. The data processingdevice may also include an output device such as a printer 1210, cameraflash, LED, haptic feedback, speaker, vibrator, NFC (Near FieldCommunication). In addition, the device may include location sensingdevices (GPS etc.), as well as clocks and other circuitry (not shown).

As shown in FIG. 13, embodiments of the subject matter described in thisspecification can be implemented in a computing system 1300 thatincludes a back-end component, e.g., as a data server, or that includesa middleware component, e.g., an application server, or that includes afront-end component, e.g., a client computer 1200 having a graphicaluser interface or a Web browser 1394 through which a user can interactwith an implementation of the subject matter described in thisspecification, or any combination of one or more such back-end,middleware, or front-end components. The components of the system can beinterconnected by any form or medium of digital data communication,e.g., a communication network. Examples of communication networksinclude a wired or wireless local area network (“LAN”) and a wide areanetwork (“WAN”), an inter-network 1310 (e.g., the Internet), andpeer-to-peer networks (e.g., ad hoc peer-to-peer networks).

The computing system can include any number of clients and servers. Aclient and server are generally remote from each other and typicallyinteract through a communication network. The relationship of client andserver arises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other. In someembodiments, a server 1350 transmits data (e.g., an HTML page) to aclient device (e.g., for purposes of displaying data to and receivinguser input from a user interacting with the client device). Datagenerated at the client device (e.g., a result of the user interaction)can be received from the client device at the server. In the embodimentshown in FIG. 13, the server computer 1350 operates server enginesoftware 1360 and web management software 1370 to receive data from andsend data to remote clients. In addition, the server computer operates adatabase 1392 to store persona information for users who wish to receiveads as described above. Content management software 1380 and databasemanagement software 1390 allow the server computer to store and retrievepersona information from the database and to search the database forpersonas that meet advertiser's criteria for a target audience.

From the foregoing, it will be appreciated that specific embodiments ofthe invention have been described herein for purposes of illustration,but that various modifications may be made without deviating from thespirit and scope of the invention. Accordingly, the invention is notlimited except as by the appended claims.

Appendix A

Appendix A serves to briefly discuss concepts disclosed in theincorporated patent applications including: brand sorting interfaces,the relationship between brand sorting and marketing data,creating/supplementing profiles (advertars) from brand sorting andmarketing data, taxonomies among some concepts.

Personas/Profiles/Advertars

In one embodiment, the demographic characteristics attributed to apersona are determined based on responses to the user's indicatedopinions such as likes or dislikes of a number of brands. As usedherein, characteristics may include the demographic characteristics of apopulation such as (gender, age, location, marital status etc.) as wellas properties, characteristics or traits relating to single individualusers such as a user's individual interests.

Personas can be created in any number of ways. For example, a user cancomplete a questionnaire by responding to questions regarding the user'sgender, age, income level, residence, political affiliation, musicaltastes, likes or dislikes (e.g., interest keywords), pieces of content(e.g., pictures) she likes/dislikes and so forth. Such a questionnairecan be provided on a software application (e.g. an app) that runs on auser's computing device or on a designated web site. The answersprovided to the questionnaire are converted to one or more likelydemographic characteristics that advertisers can use when choosing atarget audience for their ads. Characteristics such as demographiccharacteristics allow the advertisers to search the personas to findthose personas of users that meet the criteria of the advertiser'starget audience. Ads can then be sent to the addresses or identifiersassociated with each of the personas.

In another embodiment, personas are defined in a way that infers theuser's demographics based on the user's opinions of (e.g., how the userlikes or dislikes) various brands which may supplement or even replacethe above techniques of gathering data.

Brand Sorting/Marketing Data/Brand-Ad Matching Calculations/AudienceCreation

Operation 1 in FIG. 14 illustrates a method by which a user can indicatetheir opinion of a brand such as if they like a brand either more orless or feel neutral about the brand. As used herein, an opinion mayencompass input from any user interaction with or relating to the brand.Such examples include if a user likes/dislikes, purchase/would notpurchase, want/do not want as well as if a user is “following” a brandsuch as following a brand via Twitter™. In the embodiment shown, theinterface screen is divided into three areas (which may the interface ona mobile device touch screen, or other computing device). A neutral area(middle row) represents a neutral feeling about the brand (orunfamiliarity with the brand). Top row is an area where the user placesicons representing the brands they like more/shop at while bottom row isan area into which the user places the icons that represent the brandsthey like less or does not shop at.

In one implementation, each persona is associated with one or more tagsrepresenting different characteristics such as different demographiccharacteristics. The association may be determined via the brand sortingduring persona creation. A tag may store or be associated with a valuethat represents the likelihood (e.g., a probability distribution) thatthe demographic characteristic represented by the tag is applicable to auser. For instance, the value of the tag may reflect a probability thatthe user is male while another tag represents the likelihood that theuser lives in New York.

Based on the user's indication of their opinion of the brands, such asif each brand is liked or disliked, the tag values can be combined intoa composite value that reflects that likelihood that the user has aparticular demographic characteristic.

In one embodiment, the composite demographic information is created fromthe group of brands that are sorted by the user based on her opinions ofthe brands. In the example shown in FIG. 14, a user indicates that theyshop for (e.g. like) brands 1, 2 and 4. The user has indicated that theydon't shop for (e.g. don't like) brand 6 and are neutral towards (e.g.don't like or dislike or are unfamiliar with) brands 3, 5, 7, and 8. Inone embodiment, the tag values representing the likelihood that a userhas a particular demographic characteristic are combined depending on ifthe brand is liked or disliked. In other embodiments, buy/not buy, wouldbuy/would not buy, use or would use, do not or would not use as well asother opinions or impressions can be presented alone or in combination.

In one embodiment of the disclosed technology, the tags for the brandsrepresent the same demographic characteristic. For example, Tag 1 forall the brands may represent the likelihood that the user is a malebetween ages 25-40, while Tag 2 may represent the likelihood that theuser is a male between ages 40-55. Tag 3 may represent the likelihoodthat the user is a woman between ages 18-22 etc. Each tag has or isassociated with a value representing the likelihood of a user having adefined demographic characteristic. These values for the tags aretypically determined from information gathered from consumers whovolunteer information about themselves and what brands they like,purchase etc. Such information is typically gathered from marketing datafrom consumer surveys or a variety of other data sources. The details ofassociating consumer demographic information with particular brands areconsidered to be well known to those skilled in marketing. In otherembodiments, users may assign a value to a brand by inputting the valueitself into the computing device, assigning a relative value to eachbrand and or tag (brand X given a higher preference to brand Y by givingbrand X a location assignment a screen above or to the right of brand Y)etc.

In one embodiment, the composite demographic characteristics for apersona are created by arithmetically combining the values of the tagsfor the liked and disliked brands. In the example shown, Brands 1, 2 and4 are liked so their tag values are summed while Brand 6 is disliked soits tag values are subtracted. When combined as illustrated, Tag 2 has asummed value of 4.0 (1.5 plus 1.5 minus (−1.0)). A value of 4.0 for atag may represent a strong likelihood that a user has the demographiccharacteristic defined by the tag. On the other hand, a tag with acombined value of −2.5 may provide an indication that the user probablydoes not have the demographic characteristic associated with the tag andan inference can then be made. For example, if a composite gender tagvalue suggests the user is likely not a male, an inference can be madethat the user is a likely female. A composite of the values of the brandtags across the brands (e.g., the sum of statistical probabilities oftag A across brands X to Y as seen in FIG. 7) may also be represented bya vector that is associated with the persona. Each brand tag value inFIG. 7 may be a dimension of the vector.

In one embodiment, based upon the composite demographic characteristics,the corresponding user or persona may be placed into pre-computeddemographic segments. Such pre-computed segments are typicallydetermined from marketing survey data. Once the user is assigned to thesegment, additional associated characteristics of the pre-computedsegment may be associated to the user. In addition, ads that have beenspecifically designed to target the pre-computed segment may bedelivered to the user.

In one embodiment, an ad/offer/content that a persona may be interestedin receiving may be matched with the persona based on said personavector. Typically an ad comes with tags such as coffee, sale, spa,dancing lessons etc. Here, an ad/offer's tag values may be assignedbased on marketing data taken from consumer surveys such as aprobability distribution that a certain demographic (age, sex, incomeetc.) would likely desire to receive ads with a given ad tag. Thecomposite of ad tag values represent a vector for the ad. Each of theseoffer tag values may therefore be considered as an ad vector dimension.In one embodiment, tags related to the ad tags may be assigned alongwith their associated values to aid in ad-persona matching.

Once a persona is defined, a plurality of ads can be ordered forpresentation to the user according to likely persona affinity. Bycalculating the distance between the persona vector and the ad vector,such as their distances in N tag space, ads can be ranked in order oflikely persona desire. The result of this distance calculation may be aranked list of ads in order of affinity (i.e. the distance between thevectors) for a particular persona vector. In this manner, instead offiltering out ads, a relative ranking of ads is produced. Alternately,other distances between the ad and persona vectors (or any of theircomponents) can be calculated to produce a ranking. Various othermethods of ad filtering and ad sorting to match the appropriate ads tothe persona may also be used. In some embodiments, location, pastpurchases, sale times/items, membership in customer loyalty programs,percentage off and other factors may be used to aid in adordering/selection. In one embodiment, the calculated affinity for aparticular ad is displayed to the user as stars (e.g., an ad with ahighly calculated affinity is four our of four stars etc.). In anotherembodiment, the ordering/filtering may consider the ratio of thegeographic distance to an offer and the percentage off. For instance, ifan ad is only 10% off and the distance is several hundred miles from theuser, this ad would have a lower ordering then an ad that is 90% off andone mile away from the user. Here, the distance and percentage off etc.,may be displayed to the user as well. In yet another embodiment, thepersona may keep track of ads that resulted in a purchase by theconsumer. After a purchase, the user will not be shown the ad on thepersona that made a purchase or on all her personas.

Optionally, the dimensions on the persona vector and/or the ad vectorcan be normalized by multiplying the dimension by a scalar between forinstance, zero and one, to prevent particularly strong tag dimensionsfrom skewing the results.

In some embodiments, a user may not be limited to the binary choice ofonly indicating that they like or dislike a brand. The user may bepresented with controls to indicate that they strongly like or dislike abrand based on a number of discrete levels or using a sliding scale etc.

Taxonomies

In one embodiment, once a user has created or adopted one or morepersonas, the personas are registered with a server computer that maps apersona to one or more addresses or other identifiers to which adsshould be delivered. As discussed above, the address may be an e-mailaddress, IP address, device id., web site or another logical addressthat can be used to direct ads to the user.

A selected persona defines one or more demographic characteristics (suchas interests like Thai food) that may be of interest to advertisers inselecting a target audience to receive their ads.

A taxonomy may also expand the user's interest tags. For example, theuser has rated Thai Restaurants a +6. As such, the user would probablybe interested in prepared foods in general as well as Thai foods andperhaps even travel to Thailand (this may be based on marketing data orother tools). These relationships can be from user survey information.The new tags and associated values can be assimilated into the persona.This expansion of tags provides the user the opportunity to seeadditional topics, brands, times, locations and other relatedinformation. In addition, a user may give feedback on the tag'sdesirability and associated value.

Ads may be displayed to users on the same device on which brand sortingoccurred or on multiple different devices. The ads may be shown on thesedevices within a specified amount of time or upon an event trigger suchas proximity to a merchant's store, the start of a sale, another userexpressing interest in the ad etc.

Weighting

In one embodiment, the demographic information associated with a personais refined depending on how the user reacts to ads delivered to thepersona or previous brand sortings. For example, if the user indicatesthat they do not like an ad, one or more tag values associated with thepersona may be adjusted. In this way, a persona's determined demographiccharacteristics can be continually improved or updated. In oneembodiment, ads can be shown as icons and displayed and assignedaffinity/voted on in a manner similar to how brands are sorted asillustrated in FIG. 6. Answers such as “like the ad” “neutral” and“dislike the ad”, a picture of a “thumbs up” and “thumbs down” may bedisplayed on various screen areas so the user may know where to drag theicons to and thereby assign affinity to the ad.

In one embodiment, the feedback from user assigned ad affinity may makevery granular adjustments to a persona. In one embodiment, a simple voteon an ad may modify a plurality of aspects of a persona by consideringthe specific tag, subcategory tag and associated weights among otherthings. If an ad was assigned a negative affinity, the tag andassociated values may play a lessor role in assigning ads in the future.

System for Delivering Ads to Personas

FIG. 16 illustrates an exemplary system 1600 for creating personas andad serving to a persona on a computing device. At 1602 a mobile deviceis shown. On the screen are images representing four personas tied to asingle account. A user may optionally register the account under anyidentifier including an email address. In one embodiment, the emailaddress is one way hashed and discarded after the hash. The hash isoptionally stored by the audience engine and serves as an identifier.This prevents the storage of user's identifying information on non-userdevices and enables the user to have an identifier in case she forgetsher password etc. In another embodiment, only one persona is created andno identifier is asked from the user. Instead, a software install ID orother identifier is used to identify the persona.

A persona may be created by optionally choosing a name for the persona,associated interests/keywords (e.g. to help focus ad searches), socialmedia accounts to tie the persona to and active locations/times thepersona should be active among other parameters. Then, a brand sortingscreen may be displayed at 1604. Upon sorting a number of brands, at1606 and 1608 the brands that define the persona are transmitted to anaudience engine 1610, which may be on a remote server.

The persona's demographic characteristics are matched with ads, offers,coupons, services, products, content recommendations or other similarthings. Typically, the audience engine 1610 is in communication with athird party ad server and/or ad bidding system (not shown). The ads maybe pre-downloaded to the audience engine 1610 and analyzed. Analysis maybe performed by assigning tags and associating statistical probabilitiesthat particular demographics would be interested in the ads or assigningprobabilities to existing tags or other data related to the ad. The adsare then optionally ordered in relevance to the characteristics of aparticular persona's vector as previously discussed. Here, in responseto the persona creation, a plurality of ads are pushed to the mobiledevice at 1612 from the audience engine 1610. The ads are pushed into alocal ad server 1616 on the user's computing device. Here the local adserver is within the application 1214 that created the persona. Withinthe application 1614, is an ad tracker 1618 with a ticket book. Eachticket may be used to request an ad from an in-application persona API1622. In one embodiment, a ticket may contain information to display anad to one or more personas and/or to different devices or applicationsassociated with the persona.

The request for an ad may occur upon a user or a software request or onthe occurrence of an event such as an arrival of the device at aphysical location, keyword in communication, predetermined by anadvertiser, event on a calendar, time of a TV show, a triggering eventsuch as visiting a website, date of a product sale etc. API 1622 maystart the ad request at 1624, which is transmitted to ad tracker 1618.Ad tracker 1618 returns a return ad ticket at 1620 to API 1622. API 1622then submits the ad ticket and application ID at 1626 to the local adserver 1616. The local ad server then displays the ad on the device orother connected devices at 1628. In one embodiment, the application IDat 1626 can be directed toward other applications on a plurality ofconnected devices in order for an ad to be shown on other devices.Optionally, upon display of the ad, at 1626 a request can be made to aconnected device to display other content such as a website related tothe displayed ad or the ad itself on other devices.

APPENDIX B Sample Code for A Swote Tool: $(‘body’).mousemove(function(e){  if( keyFlag &&   false == mouseFlag) {   var offset, nearestEl, el =e.target;   //   // don't try to highlight the highlighter. Get theelement   // underneath the hightlighter   if (el.className === ‘outer’){    box.hide( );    el = document.elementFromPoint(e.clientX,e.clientY);   }   //   // get the coordinates of the element under themouse and   // use this to find the nearest img or anchor   offset =$(el).offset( );   nearestEl = $.nearest({    x: offset.left,    y:offset.top   }, ‘img, a’)[0];   //   // make common the same as nearestfor the moment. We might   // fix it up in a sec. Common is going to bethe lowest   // common ancestor of the hovered element and the nearestimg/a   common = nearestEl;   //   // if the element   // we arehovering over is different than the nearst image   // or anchor then tryto find the lowest common ancestor.   // BUT, don't do any of this ifthe element we are   // hovering over is the BODY tag. BODY doesn't havea parent   if ($(el).get(0).tagName != ‘BODY’ && el != nearestEl) {   common = commonAncestor(el, nearestEl);    //    // this can end upundefined    // if it does, just use the nearest    // img or anchor asour selection    if (typeof common == ‘undefined’) common = nearestEl;  }   //   // make the highlight box the size of the hovered element  // and put it in the correct spot;   offset = $(common).offset( );  box.css({    ‘width’: $(common).outerWidth( ) − 1,    ‘height’:$(common).outerHeight( ) − 1,    ‘left’: offset.left,    ‘top’:offset.top   });   //   // it's in the right spot, so show it  box.show( );  } });

I/We claim:
 1. A non-transitory computer readable media havinginstructions stored thereon that are executable by processor electronicsto: display one or more items of content to a user; detect a selectionby the user of at least one item of content displayed; in response tothe selected content, determining if the selected content includes anyrelated information; and transmitting a reference to an identity of theselected content and any related information to a remote computer systemto use the selected content and related information to update a profileof likely characteristics of the user based on the selected content andthe information related to the selected content.
 2. The non-transitorycomputer readable media of claim 1, further comprising instructions thatdetermine a link associated with the selected content.
 3. Thenon-transitory computer readable media of claim 1, further comprisinginstructions that assign a tag to the selected content wherein the tagis a record of the gesture.
 4. The non-transitory computer readablemedia of claim 1, further comprising instructions to indicate that oneor more items of content on the display are selectable with a usergesture as selected content.
 5. A non-transitory computer readable mediahaving instructions stored thereon that are executable by processorelectronics to: tag an item of content with one or more likelycharacteristics that can be attributed to persons that react to the itemeither favorably or unfavorably; display the content for a user; detecta gesture performed by the user directed towards the content; and updatea profile that indicates one or more likely characteristics of the userbased on the detected gesture directed towards the content.
 6. Thenon-transitory computer readable media of claim 5, further comprisinginstructions that cause the processor electronics to detect the gesturedirected towards the content by: detecting when the user touches thecontent on a display screen; and detecting when the user moves a touchpoint on the display screen in a predefined direction over the content.7. The non-transitory computer readable media of claim 6, furthercomprising instructions that cause the processor electronics tore-display the selected content at a location that is under the touchpoint of the user as the user is moving the touch point on the displayscreen while performing the gesture.
 8. The non-transitory computerreadable media of claim 7, further comprising instructions that causethe processor electronics to display indicia of whether the user isindicating a favorable or unfavorable impression of the content in aspace where the content was first displayed prior to the content beingre-displayed as the user is performing the gesture.
 9. Thenon-transitory computer readable media of claim 7, further comprisinginstructions that cause the processor electronics to: determine anangular direction at which the user performs the gesture as measuredfrom a point where the user first touches the content; and perform acommand on the content if the angular direction is not within a rangedefined as providing a favorable impression of the content or a rangedefined as providing an unfavorable impression of the content.
 10. Thenon-transitory computer readable media of claim 5, further comprisinginstructions that cause the processor electronics to display arepresentation of whether the processor electronics recorded a favorableor unfavorable impression of the content after the user has finished thegesture.
 11. The non-transitory computer readable media of claim 5,further comprising instructions that cause the processor electronics todisplay a menu over the content that indicates what command will beinterpreted by the processor electronics depending on an angulardirection in which the gesture is performed.
 12. The non-transitorycomputer readable media of claim 5, further comprising instructions thatcause the processor electronics to assign the tag to the content basedat least upon a URL or URI of a website from where the content wasobtained.
 13. The non-transitory computer readable media of claim 6,further comprising instructions that cause the processor electronics toassign the tag to the content based at least upon other contentpresented with the content.
 14. The non-transitory computer readablemedia of claim 12, further comprising instructions that cause theprocessor electronics to transmit a record of the gesture detected onthe content to another computing device.
 15. The non-transitory computerreadable media of claim 11, further comprising instructions that causethe processor electronics to recommend content to a user based at leaston the updated characteristics associated with the user performing thegesture on the content.
 16. The non-transitory computer readable mediaof claim 5, further comprising instructions that cause the processorelectronics to update the profile by receiving one or more of: GPScoordinates indicative of a location of the user, a time, a date, a userID, a user browsing history, or user inputs.
 17. The non-transitorycomputer readable media of claim 5, further comprising instructions thatcause the processor to add the characteristics represented by the tag toa user's interest graph.
 18. The non-transitory computer readable mediaof claim 17, further comprising instructions that cause the processor togather a plurality of user interest graphs having substantially similardemographic characteristics represented by similar tags.
 19. Anon-transitory computer readable media containing instructions that areexecutable by processor electronics to display content on a displayscreen by: displaying items of content on a display screen; andpresenting an indication associated with one or more of the items ofcontent that indicates that a user may perform a gesture on the item ofcontent by moving the item of content in a predefined direction on thedisplay screen in order to indicate that the user likes or dislikes theitem of content.
 20. The computer readable media of claim 19, furthercomprising instructions that are executable by the processor electronicsto detect the gesture by: determining if the user has moved a point ofcontact over the item of content in a predetermined direction.